To manhandle or beat up
The gangsters roughed him up a little.
To make rough, to roughen (especially doing so to wood while raising the grain)